1. Field of the Invention
This invention relates in general to mass storage systems, and more particularly to a method and apparatus for emulating shared memory in a storage controller.
2. Description of Related Art
Modern mass storage subsystems are continuing to provide increasing storage capacities to fulfill user demands from host computer system applications. Due to this critical reliance on large capacity mass storage, demands for enhanced reliability are also high. Various storage device configurations and geometries are commonly applied to meet the demands for higher storage capacity while maintaining or enhancing reliability of the mass storage subsystems.
As it is known in the art, computer systems generally include a central processing unit, a memory subsystem and a storage subsystem. According to a networked or enterprise model of a computer system, the storage subsystem associated with or in addition to a local computer system, may include a large number of independent storage devices or disks housed in a single enclosure. This array of storage devices is typically connected to several computers (or hosts) via dedicated cabling or via a network. Such a model allows for the centralization of data which is to be shared among many users and also allows a single point of maintenance for the storage functions associated with the many computer systems.
One solution to these mass storage demands for increased capacity and reliability is the use of multiple smaller storage modules configured in geometries that permit redundancy of stored data to assure data integrity in case of various failures. In many such redundant subsystems, recovery from many common failures can be automated within the storage subsystem itself due to the use of data redundancy, error correction codes, and so-called “hot spares” (extra storage modules which may be activated to replace a failed, previously active storage module). These subsystems are typically referred to as redundant arrays of inexpensive (or independent) disks (or more commonly by the acronym RAID). The 1987 publication by David A. Patterson, et al., from University of California at Berkeley entitled A Case for Redundant Arrays of Inexpensive Disks (RAID), reviews the fundamental concepts of RAID technology.
There are five “levels” of standard geometries defined in the Patterson publication. The simplest array, a RAID level 1 system, comprises one or more disks for storing data and an equal number of additional “mirror” disks for storing copies of the information written to the data disks. The remaining RAID levels, identified as RAID level 2, 3, 4 and 5 systems, segment the data into portions for storage across several data disks. One of more additional disks are utilized to store error check or parity information.
RAID storage subsystems typically utilize a control module that shields the user or host system from the details of managing the redundant array. The controller makes the subsystem appear to the host computer as a single, highly reliable, high capacity disk drive. In fact, the RAID controller may distribute the host computer system supplied data across a plurality of the small independent drives with redundancy and error checking information so as to improve subsystem reliability. Frequently RAID subsystems provide large cache memory structures to further improve the performance of the RAID subsystem. The cache memory is associated with the control module such that the storage blocks on the disk array are mapped to blocks in the cache. This mapping is also transparent to the host system. The host system simply requests blocks of data to be read or written and the RAID controller manipulates the disk array and cache memory as required.
To further improve reliability, it is known in the art to provide redundant control modules to reduce the failure rate of the subsystem due to control electronics failures. In some redundant architectures, pairs of control modules are configured such that they control the same physical array of disk drives. A cache memory module is associated with each of the redundant pair of control modules. The redundant control modules communicate with one another to assure that the cache modules are synchronized. When one of the redundant pair of control modules fails, the other stands ready to assume control to carry on operations on behalf of I/O requests. However, it is common in the art to require host intervention to coordinate failover operations among the controllers.
It is also known that such redundancy methods and structures may be extended to more than two control modules. Theoretically, any number of control modules may participate in the redundant processing to further enhance the reliability of the subsystem.
However, when all redundant control modules are operable, a significant portion of the processing power of the redundant control modules is wasted. One controller, often referred to as a master or the active controller, essentially processes all I/O requests for the RAID subsystem. The other redundant controllers, often referred to as slaves or passive controllers, are simply operable to maintain a consistent mirrored status by communicating with the active controller.
The disk devices of a storage system are typically configured to represent one or more so called logical device. A logical device is a way to define a contiguous area of storage space as being available as a distinct addressable unit. The addresses used to access data in a logical device typically need to be translated into physical addresses in order to find the requested data. In many systems, a logical device includes all the addressable storage of a single physical volume (e.g. disk drive). As taught in the prior art, for any particular RAID logical unit (LUN—a group of disk drives configured to be managed as a RAID array), there is a single active controller responsible for processing of all I/O requests directed thereto. The passive controllers do not concurrently manipulate data on the same LUN.
It is known in the prior art to permit each passive controller to be deemed the active controller with respect to other LUNs within the RAID subsystem. So long as there is but a single active controller with respect to any particular LUN, the prior art teaches that there may be a plurality of active controllers associated with a RAID subsystem.
Information and work must be easily and quickly shared between controllers in a loosely coupled dual controller storage subsystem. The controllers are loosely coupled, as the only means of communication is over Fibre Channel connections that are shared with the physical disk drives. These Fibre Channel connections provide only asynchronous communication with high latency. Furthermore, it is tedious and error prone to use the Fibre Channel connections. The programmer must setup buffers for DMA activity, processes for sending data, locking mechanisms to maintain coherency on the shared data, plus failover and failback considerations. Every time a new feature is added that requires work or information be shared between the two controller, many of these pieces are re-invented to fit the exact task that must be performed.
A proven method for easily sharing work and information between multiple processors in a storage subsystem is through the use of shared memory. Shared memory in storage controllers is typically a bank of memory located on its own card connected by low latency busses to a number of processors in the storage system. The processors may have direct memory mapped access to the shared memory. Data that is placed in the shared memory is used to share work between the processors in the system. This shared data is usually under lock control, where rules that programmers must follow govern access to the data based on acquiring the lock.
Some controllers do not have a shared memory, but rather communicate over Fibre Channel connections that are shared with the physical disk drives. It is tedious to use the Fibre Channel connection to share data and work between the two controllers. The programmer must setup buffers for DMA activity, processes for sending data, locking mechanisms to maintain coherency on the shared data, plus failover and failback considerations. Every time a new feature is added that requires work or information be shared between the two controller, many of these pieces are re-invented.
It can be seen that there is a need to emulate shared memory in a storage controller.
It can also be seen that there is a need to emulate shared memory in a storage controller to prevent reinventing firmware when new features are added to a storage controller.